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5 (57) Abstract: The present invention is a system and a method for providing a control unit Tor a multipoint multimedia/audio con- 
2 ference that enables one or more participants to take part in more than one conference simultaneously. The control unit can operate 
® in audio and/or video sections of an MCU and/or Management and Control System (MCS). The MCS, in an exemplary embodiment 
Q of the present invention, controls the participation of al least one participaniin more than one conference simultaneously by using a 
J>> Cross-Conference Database (CCDB). The MCU performs connection changes atfecting which participants are associated with one 
^ or more conferences based on the information that is stored in theCCDB. _ m t BA ,„ _ _ n 
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I CONTROL UNIT FOR MULTIPOINT MULTIMEDIA/AUDIO SYSTEM 
2 

3 BACKGROUND OF THE INVENTION 

4 Field of the Invention 

5 [0001] The present invention relates generally to conferencing systems, and more 

6 particularly to a control unit for a multipoint audio and or multimedia conference. 
7 

8 Description of Prior Art 

9 [0002] In current audio conferences or videoconferences, a participant is an entity that can 
10 participate in a single conference (i.e., the participant can speak and/or listen to a single 

II conference). However, often a participant needs to speak to one group and during the same 

1 2 time may need to listen to another group of related or unrelated conferences. 

13 [0003] An example for such a need is a conference with two groups of people who do not 

14 speak the same language and need translators. For instance, one such scenario includes a group 

15 having English speaking individuals, El to En, a second group having French speaking 

16 individuals, Fl to Fm, a translator from English to French (EF) and a translator from French to 

17 English (FE). In such a case, the participants of the conference may desire to have the 

18 following related conferences: conference A with participants El to En, Fl to Fm, and FE and 

19 conference B with participants EI to En, Fl to Fm, and EF. 

20 [0004] The French-speaking individual may want to listen to conference B and the English 

21 speaking individuals may want to listen to Conference A. As a result, only the English 

22 speaking individuals will hear the FE translator, and the French speaking individuals will hear 



WO 02/091641 



PCT/1L02/00361 



1 the EF translator. Another example in which participants of a conference may desire to have 

2 more than one conference is if some of the participants want a "Private Room" (a virtual private 

3 room) in which a first group of participants can hear a second group of participants, while 

4 chatting among themselves without being heard by the second group of participants. 

5 {0005] A Multipoint Control Unit (MCU) is a conference controlling entity. The MCU may 

6 be a piece of equipment located in a node of the network or in a terminal that receives several 

7 channels from access ports and, according to certain criteria, processes audio or audiovisual 

8 signals and distributes them to connected channels. The audio signals are processed according 

9 to signaling protocol in the circuit switch or packet switched networks such as, but not limited 

10 to, Public Switched Telephone Network (PSTN), Integrated Services Digital Network (ISDN), 

11 Asynchronous Transfer Mode (ATM), Internet Protocol (IP), Session Initiation Protocol (SIP), 

12 H.320, H.323, or a similar protocol. An example of an MCU is MGC-100, which is available 

13 from Polycom, Inc. When the MCU is for an audio/multimedia conference, the MCU may 

14 process the audio signals and distribute them to connected channels, and may be used for 

15 controlling a conference. However, current MCUs cannot fulfill the needs of those individuals 

16 desiring to participate in multiple conferences such as the examples given above. 

17 [0006] Therefore, it is evident that there is a need for a system and a method, which enables 

1 8 one or more participants to take part in more than one conference. 
19 
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1 SUMMARY OF THE INVENTION 

2 [0007] The present invention provides a system and a method for controlling an audio or 

3 multimedia conference that enables each participant to participate in more than one conference, 

4 The present invention may operate in an audio section and in a Management and Control 

5 System (MCS) section of an MCIL 

6 In the present invention, a bridge may be a logical unit that is identified with a 

7 conference. The bridge may include a stream analyze and enhance logical unit, a control unit, a 

8 switch, and a mixer. The analyze and enhance logical unit may include a set of algorithms 

9 analyzing an audio stream of a participant and enhancing its quality. The analysis and 

10 enhancement may include, but is not limited to, ITU G.165 (Echo canceling), DTMF 

11 suppression, Voice Activity Detection (VAD), for example. The control unit may be a main 

12 control unit for the conference, and may receive all information signals from the stream analyze 

13 and enhance logical units. The control unit may select the participants that will be routed for 

14 mixing. The switch may be a selector that receives the decoded streams from all the 

15 participants in a conference and transfer the selected streams to a mixer. The selection is based 

16 on the decisions of the control unit. The mixing unit receives the streams from all of the 

17 selected participants and supplies each participant with an uncompressed mixed audio stream 

18 from die selected participants. A connection parameter may indicate resources that are 

19 allocated to each participant and to each conference. For example, the connection parameter 

20 may be a parameter related to a codec that is associated with the participant, and/or a bridge that 

21 is associated with the conference. In course of the description, words such as compress and 

22 encode may be used interchangeably. Similarly, as the phrases decode, uncompress and open 
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1 data may be used interchangeably. 

2 [0008] Further in the present invention, a Cross-Conference Database (CCDB) may be a 

3 management tool that enables participation of at least one participant in two or more 

4 conferences simultaneously. The CCDB is a database that holds connection statuses of each 

5 participant in all the conferences that are currently managed by the MCU and/or the connection 

6 parameters of all the conferences and/or all the participants that are currently connected to the 

7 MCU. The connection status may define the status of the participant in a conference. Some 

8 examples of connection statuses are Normal, Mute, Force, Speak, Listening, and None. The' 

9 Mute (M) connection status may mean that the participant cannot be heard in the conference. 

10 The Normal (N) connection status may mean that the participant can be heard and can listen to 

11 the conference. The Speak (S) connection status may mean that a participant can be heard in 

12 the conference but cannot listen to the conference. Optionally, for an N and/or S connection 

13 status the participant may be heard only if the signal representing the participant's voice meets 

14 certain criteria such as whether the energy level is above a certain value. The Force (F) 

15 connection status may mean that the participant must be heard in the conference even if the 

16 corresponding participants voice does not meet the criteria for being heard. The F connection 

17 status may also allow the participant to listen to the conference. The Exclusive (E) connection 

18 status may mean that the participant is the only one that can be heard in the conference. The 

19 Listening (L) or Mute (M) connection status may mean that the participant can listen to the 

20 conference without speaking. Finally, the None connection status may mean that the participant 

21 has no relations to this conference. 
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1 [0009] The MCS of the present invention may control the participation of at least one 

2 participant in more than one conference by using the Cross Conference Database (CCDB). The 

3 MCU may perform connection changes of participants or conferences based on information that 

4 is stored in the CCDB. A connection change may be any change in the current situation. A 

5 connection change can be, for example, the start of a conference, the termination of a 

6 conference, the addition of a participant to the conference, or the muting of a participant. The 

7 CCDB can be implemented in a single database or in several databases. For example, there 

8 may be a database for each participant that may include the connection status and/or the 

9 connection parameters of that participant in every conference that is currently controlled and/or 

10 managed by the MCU. As another example, there may be a database for each conference that is 

11 currently controlled by the MCU, where the database may include the connection status of all of 

12 the participants in the conference and/or the connection parameters of the conference. 

13 [0010] In one embodiment, the MCS may have one or more routines that manage the effect 

14 of changes in one conference on some of or all of the other conferences, and/or the effect of 

15 changes in the connection status of one participant on some of or all of the conferences. In this 

16 application, an encoder may be an enhancement and encoding logical unit, compressing the 

17 audio signal for participants, based on the communication standard, such as, but not limited to, 

18 G.723.1, G.728, G.729, or MPEG. A decoder may be a logical unit for decoding a compressed 

19 audio stream, based on the communication standards like but not limited to: G.723.1, G.728, 

20 G.729, MPEG. The word "codec" refers to a unit that may be a logical unit and includes a 

21 decoder or decoding section and an encoder or encoding section. Also in this application, the 

22 word "bridge" refers to a unit, which may be a logical unit, that is associated with the 
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1 conference. The audio section of an embodiment of the present invention has an architecture 

2 that enables multicasting decoded output from each participant's codec to the bridges of all the 

3 conferences. Moreover, the audio section may enable routing of the output of any bridge or any 

4 of a group of bridges to any participant or any of a group of participants. 

5 [0011] Other objects, features, and advantages of the present invention will become 

6 apparent upon reading the following detailed description of the embodiments with the 

7 accompanying drawings and appended claims. 
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1 BRIEF DESCRIPTION OF THE DRAWINGS: . 

2 [0012] FIG. 1 A is a block diagram showing a conference environment; 

3 [0013] FIG. IB is a block diagram of an embodiment of the invention including a 

4 general description of an audio unit; 

5 [0014] FIG. 1C is a flowchart of a method of operation for the embodiment of FIG. IB; 

6 [0015] FIG. 2 is an example of a Cross-Conferences Database (CCDB); 

7 [0016] FIG. 3 is a flow diagram showing the steps of an exemplary method for 

8 disconnection of a participant; and 

9 [0017] FIG. 4 is a flow diagram showing the steps of an exemplary method for 
1 0 terminating a conference. 
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1 DESCRIPTION OF EXEMPLARY EMBODIMENTS 

2 [0018] Referring now to the drawings, in which like numerals refer to like parts 

3 throughout the several views, exemplary embodiments of the present invention are 

4 described. 

5 [0019] FIG. 1A is an exemplary block diagram illustrating a general description of a 

6 conference environment 100 having endpoints lllOaa-nk, operator 1115, multimedia 

7 signals 1120aa-nk, multimedia communications 1122a-k, networks 1130a-k, and 

8 Multimedia Conference Control Unit (MCCU) 1140. In one exemplary embodiment, the 

9 MCCU 1140 may include a Network Interface (NI) 1142, Compressed Audio Common 

10 Interface (CACI) 110, audio unit 1160, Management and Control System (MCS) 1170, 

11 control signals 1174, a host 1200, and video unit 1300. Other exemplary embodiments may 

1 2 not have a video section and may be used for audio conferences only. 

13 [0020] The pluralities of endpoints lllOaa-nk are connected via the plurality of 

14 networks 1 130a-k to the MCCU 1 140. The MCCU 1140 may be an MCU, or an audio only 

15 multipoint control unit (an audio bridge), for example. The MCCU 1 140 and/or some or all 

16 of its components are logical units that may be implemented by hardware and/or software. 

17 The MCS 1170 may be a control module and may be a logical unit that controls the 

18 operation of the MCCU 1 140. 

19 [0021] An endpoint is a terminal on a network, capable of providing one way or two- 

20 way audio and/or visual communication with other terminals or with the MCCU 1440. The 

21 information communicated between the terminals and/or the MCCU 1440 may include 

22 control signals, indicators, audio information, video information, and data. A terminal may 

23 provide any combination of several different types of inputs and/or outputs, such as speech 
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1 only, speech and data, a combination of speech and video, or a combination of speech, data, 

2 and video. 

3 [0022] The NI 1142 receives multimedia communications 1122a-k via a plurality of 

4 networks 1130a-k and multimedia signals 1120aa-nk from the plurality of the endpoints 

5 1110aa-nk, and processes the multimedia communication according to communication 

6 standards that are used by each type of network, such as, but not limited to, H.323, H.321, 

7 SIP, and/or H.320. The NI 1 142 then delivers compressed audio, compressed video, 

8 compressed data, and control streams to appropriate logical modules in the MCU 1140. 

9 Some communication standards require that the process of the NI 1142 include 

10 demultiplexing the incoming multimedia communication into compressed audio, 

11 compressed video, compressed data and control streams. In the opposite direction, the NI 

12 1142 receives the separate streams from the various units (e.g., the MCS 1170, audio unit 

13 1160, and/or video unit 1300) and processes the streams according to the appropriate 

14 communication standard. The NI 1142 then transmits the streams to the appropriate 

15 network 11 30a-k. 

16 [0023] The audio unit 1160 receives the compressed audio streams of the plurality of 

17 endpoints 1 1 1 Oaa-nk via the NI 1 142 and CACI 1 10, processes the audio streams, mixes the 

18 relevant audio streams, and sends the compressed mixed signal via the Compressed Audio 

19 Common Interface (CACI) 110 and the NI 1142 to the endpoints lllOaa-nk. Audio unit 

20 1 1 60 may be a logical unit and is described below in conjunction to Fig. 1 B . 

21 [00241 The video unit 1300 may be a logical unit that receives and sends compressed 

22 video streams. The video unit 1300 includes at least one video input module that handles an 

23 input portion of a video stream 1302 from a participating endpoint and at least one video 

24 output module that generates a composed compressed video output stream that is sent via 
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1 Compressed Video Common Interface (CVCI) 1302 to NI 1142 and from there to the 

2 designated endpoints 111 Oaa-nk. 

3 [0025] The uncompressed video data is shared by input and output modules on a 

4 common interface such as, but not limited to, Time Division Multiplexing (TDM), 

5 Asynchronous Transfer Mode (ATM), and/or shared memory. The data on the common 

6 interface may be fully uncompressed or even partially compressed. An exemplary operation 

7 of such a video unit is described in U.S. Patent Number U.S. 6,300,973, which is 

8 incorporated herein by reference. 

9 [0026] Preferably, the host 1200 communicates with the operator 1115 of the MCCU 

10 1 140, where the operator 1115 may have an operator's station for communicating with the 

11 host 1200. The host 1200 controls the MCCU 1140 via the MCS 1170 according to 

1 2 instructions from the operator 1115. 

13 FIG. IB is an exemplary block diagram of an embodiment of the invention including a 

14 general description of audio unit 1 160. The embodiment of FIG. IB includes a Compressed 

15 Audio Common Interface (CACI) 110, a control bus 135, MCS 1170, and audio unit 1160 

16 having compressed signals 115 and 117, codec 120, decoded information 126, mixed output 

17 128, Decoded Audio Common Interface (DACI) 140, and bridge 150. The codec 120 

18 includes a decoder 122 and an encoder 124, while the bridge 150 includes analyze and 

19 enhance unit 152, information signal 153, control unit 154, switch 156, control signals 157, 

20 selected signals 159, mixer 160, and mixed signal 161. FIG. IB describes the flow of audio 

21 streams in one example of the present invention. Compressed audio streams, from all 

22 endpoints that are connected to an MCU are transferred over the Compressed Audio 

23 Common Interface (CACI) 1 10. The MCS 1170 allocates a codec 120 to each one of the 

24 endpoint 1 1 1 Oaa-nk (FIG. 1 A). 
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1 [0027] Further, the CACI 110 carries signals to and from endpoints lllOaa-nk. For 

2 example, the compressed signal 115 from one of the endpoint lllOaa-nk is routed through 

3 the CACI 110 to the decoder 122 in the codec 120, which was previously allocated to that 

4 endpoint by the MCS 1 170 via control bus 135. The decoder 122 may be a logical unit and 

5 may decode a compressed audio stream, based on the communication standards such as, but 

6 not limited to, G.723.1, G.728, G.729, MPEG. The decoder 122 then decodes the 

7 compressed audio stream, such as compressed signal 115, and broadcasts the decoded signal 

8 126 over the Decoded Audio Common Interface (DACI) 140. The DACI 140 is a bus that 

9 may have broadcasting capabilities. The DACI 140 may be implemented for example by 

10 any one of or any combination of Time Division Multiplexing (TDM), Asynchronous 

11 Transmission Mode (ATM), Local Area Network (LAN), wireless technology, or shared 

12 memory. The bridge 150 may then grab the decoded signal from the DACI 140 and may 

13 analyze, enhance, and/or mix the decoded signal and return the output 161 to the DACI 140. 

14 [0028] The encoder 124 may be a logical unit and may be an enhancement and/or 

15 encoding unit. The encoder 124 may compress the output 128 of the appropriate bridge 150 

16 forming a compressed audio stream, such as the compressed signal 117, based on the 

17 communication standard such as, but not limited to, G.723.1, G.728, G.729, and/or Motion 

1 8 Picture Expert Group (MPEG). 

19 [0029] The MCS 1170 generates a Cross-Conferences Database (CCDB) based on the 

20 required setup of all the participants and all the conferences that currently exist in the MCU. 

21 The CCDB is a Cross-Conference Database that holds the connection parameters (e.g., 

22 codecs and bridges, etc.) and the connection status (e.g., Normal, Mute etc.) of each 

23 endpoint (participant) that is currently connected to the MCU, in every conference that is 

24 currently managed by the MCU. The CCDB enables the participation of at least one 

ii 
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1 participant in more than one conference. The CCDB is described below in conjunction with 

2 Fig 2. According to the CCDB, the MCS 1 170 programs one or more bridges 150 to grab 

3 from the DACI 140 the decoded signals of all the participants associated with a conference 

4 that is assigned to those bridges 150. 

5 [0030] The decoded output of any codec 120 can be grabbed by more than one bridge 

6 150 allowing the participants to be associated with more than one conference. The decoded 

7 streams from the decoders 122 on the DACI 140 may be grabbed by the bridge 150 and then 

8 analyzed and enhanced by the analyze and enhance unit 152. The analyze and enhance unit 

9 152 may be a logical unit, and may include a set of algorithms for analyzing an audio stream 

10 of a participant and/or enhancing its quality, such as, but not limited to, International 

11 Telecommunications Union (ITU) G.165 (echo canceling), Dual Tone Multi-Frequency 

12 (DTMF) suppression, and/or Voice Activity Detector (V AD). 

13 [0031 J The bridge 150 may have one or more analyze and enhance units 152. Each 

14 analyze and enhance unit 152 is assigned to a single participant and is programmed 

15 according to the connection status of that participant in the conference. The control unit 154 

16 controls a conference that receives all signals from the analyze and enhance unit 152 and 

17 selects the participants that will be routed via switch 156 to the mixer 160. The mixer 160 

18 receives the enhanced streams from all of the selected participants and supplies each 

19 participant with an uncompressed mixed audio stream of the selected participants. 

20 Signals from the analyze and enhance unit 152 are sent to the control unit 154 and the 

21 enhanced decoded audio signals are sent from the analyze and enhance units 152 to the 

22 switch unit 156. The switch unit 156 is a selector that receives the decoded streams from all 

23 the participants in a conference and transfers the selected streams to the mixer 160. The 

24 selection is based on the decisions of the control unit 154. Based on received commands 
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1 from the MCS 1 170, which define the connection status of the participants in the conference 

2 that is assigned to the bridge 150, and the information signal 153 from the analyze and 

3 enhance unit 152 the control unit 154 controls, via control signals 157, the switch 156, and 

4 the mixer 160, For example, in a case where a participant's connection status is Normal (N), 

5 the analyze and enhance unit 152 that is associated with that participant may indicate that 

6 the voice signal meets a certain criteria such as set forth by VAD, (e.g., such as the energy 

7 level being above a certain value.). Then, the control unit 154 via switch 156 selects the 

8 output of the analyze and enhance unit 152, which is assigned to the participant, as one of 

9 the inputs to the mixer 160. The mixer 160 mixes the selected audio signals to form the 

10 mixed signal 161, and broadcasts the mixed signal 161 over the DACI 140. Some 

11 embodiments of the bridge 150 have the capability of eliminating the voice of a speaker 

12 from the mixed signal that is aimed to the endpoint of that speaker. 

13 [0032] The MCS 1 170, based on the connection status stored in the CCDB, commands 

14 one or more codecs 120 to grab the mixed output 128 from the DACI 140 for listening to the 

15 conference. After grabbing the mixed output 128 from the DACI 140, the encoder 124 

16 encodes the decoded signal from the appropriate bridge 150, and sends the compressed 

17 signal 1 17 via the CACI 1 10 to the appropriate participant. 

18 [0033] The codecs 120 and the bridges 150 may be implemented by Digital Signal 

19 Processors (DSPs) such as, but not limited to, Texas Instruments DSP, TMS320C31. One 

20 DSP can include more than one unit (e.g., more than one codec and/or bridge). In the above 

21 example, the codec 120 handles a single participant's audio signal, and the bridge 150 

22 handles one conference. 

23 [0034] Referring now to FIG. 1C, a flowchart depicting a method 170 for the operation 

24 of the system of FIG. IB is shown. In the method 170, the mixed signal is broadcasted back 
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1 to the endpoints 1 1 lOaa-nk (FIG. 1 A). Specifically, the method 170 starts with step 172 in 

2 which the MCS 1 170 (FIG. 1 A) receives signals from the host 1200 (FIG. 1A) or from one 

3 or more endpoints lllOaa-nk (FIG. 1A) related to the configuration of the conference or 

4 conferences in progress or that needs to be initiated. The endpoints 1 1 lOaa-nk communicate 

5 with the MCS 1170 indirectly by sending multimedia signals 1120aa-nk (FIG. 1A) to 

6 networks a-k which in turn send multimedia communications 1 122a-k (FIG 1A) to the NI 

7 1 142 (FIG. 1 A). Then, the NI 1 142 responds to the multimedia communication 1 122a-k by 

8 sending control signal 1 174 to the MCS 1 170 to set up the conference. In one embodiment, 

9 the audio unit 1160 (FIG. 1A) and/or the video unit 1300 (FIG. 1A) may send control 

10 signals to the MCS 1 170 for setting up the conference in addition to or instead of the control 

11 signals 1174 (FIG. 1A). 

12 [0035J Next, in step 174, the MCS 1 170 creates and/or updates one or more CCDBs in 

13 which the information about how the conferences are to be setup is stored, and broadcasts 

14 control signals on the control bus 135 (FIG. IB). Subsequently, the codecs 120 (FIG. IB) 

15 receive and/or grab control signals from control bus 135 associating individual codecs 120 

16 with endpoints and/or participants according to the configuration of the conferences in the 

17 CCDB in step 176. Then, in step 178, the bridges 150 (FIG. IB) grab and/or receive control 

18 signals from the control bus 135 associating each analyze and enhance unit 152 with an 

19 endpoint and/or participant Next, compressed audio signals from one or more endpoints are 

20 placed or broadcasted on the CACI 1 10 (FIG. IB) in step 180. Subsequently in step 182, 

21 the corresponding decoder 122 (FIG. IB) of the codec 120 may grab and/or receive the 

22 compressed signal 115 (FIG. IB) from the CACI 110 and decodes the compressed signal 

23 1 15 to produce the decoded signal 126 (FIG. IB) and broadcasts the decoded signal 126 on 

24 the DACI 140 (FIG. IB). 
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1 [0036] Subsequently, in step 184, the analyze and enhance unit 152 (FIG. IB) grabs 

2 and/or receives the decoded signal 141 (FIG. IB) (which may be derived from the decoded 

3 signal 126) from the DACI 140 according to control signals from the MCS 1 170, which is 

4 driven from CCDB (step 174). Also, in step 184, the analyze and enhance unit 152 

5 enhances the decoded signal 141 to form the enhanced signal 155 (FIG. IB), and extracts 

6 the control information 153 (FIG. IB). The enhanced signal 155 is then sent to the switch 

7 1 56 (FIG. IB), and the control information 1 53 is sent to the control unit 1 54 (FIG. 1 B). 

8 [0037] The control unit 154 produces control signals 157 (FIG. IB) based upon control 

9 signals from the MCS 1170 (which are driven from CCDB in step 174) and/or control 

10 information 153 in step 190. Based on the control signal 157 and/or control signals from the 

11 MCS 1170, the switch 156 selects which enhanced signals 155 are sent as selected signals 

12 159 (FIG. IB) to the mixer 160 (FIG. IB) in step 191. In response, in step 192, the mixer 

13 160 mixes the selected signals 159 to form the mixed signal 161 (FIG. IB) according to 

14 control signals from the MCS 1 170 and broadcasts the mixed signal 161 onto the DACI 140. 

15 One or more encoders 124 (FIG. IB), based on CCDB (step 174), may grab and/or receive 

16 the mixed output 128 (which may be derived from mixed signal 161), and encode the mixed 

17 output 128 to form the compressed audio signal 117 (FIG. IB) in step 194. The compressed 

18 audio signal 1 17 is then broadcasted onto the CACI 110. Finally, in step 196, the endpoints 

19 UlOaa-nk grab and/or receive the mixed compressed audio signal via the NI 1142 and 

20 networks 11 30a-k, 

21 [0038] The steps of the method 170 have been presented in an order that facilitates 

22 understanding the method. However, the steps of the method 170 can be performed in any 

23 order. Since some endpoints, according to the information stored in CCDB (step 174), may 

24 only be able to talk and some may only be able to listen, some codecs 120 (associated with a 

15 
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1 talk only endpoint) may grab and/or receive the compressed signals 115 from a particular 

2 endpoint, and broadcast the compressed signals 117 (for other endpoints to listen to), while 

3 other codecs 120 may not grab or receive the compressed signals 115 (from their associated 

4 endpoint), but may nonetheless send the mixed compressed signals 117 to their associated 

5 endpoint. 

6 [0039] FIG. 2 is an example of a CCDB. Each column represents a conference and the 

7 bridge 150 (FIG. 1A) that is associated with that conference. In FIG. 2, the participants are 

8 labeled lto N+2, and the conferences are labeled A-X. The bridges used for the conference 

9 are labeled Bl-Bz. French speaking participants are marked with an "F," and English 

10 speaking participants are marked with an "E." The English to French translator is marked 

11 with an "EF" and the French to English translator is marked with an "FE" The codecs in 

12 use are labeled COl-Ck. Cells marked V have a status of Normal, cells marked "S" have a 

1 3 status of Speak, and empty cells have a status of "None." 

14 [0040] For example, column A 202 (i.e., conference A) uses bridge # 3 (i.e., B3), and 

15 the first row 204 is participant #1 who is using codec # 1 1 (CI 1) 206. The current example 

16 presents a case with five conferences. The first conference is conference A, which uses B3 

17 as the bridge. Conference A has six participants, labeled 1, 2, 3, 4, 5, and 6, which use the 

18 codecs 120 (FIG. IB) having numbers Cll, C21, C13, C05, C07, and C06, respectively. In 

19 this conference, the connection status of the participants 1, 3, 5 (the English speaking 

20 individuals) and 6 (French to English translator) is "Normal" (N). The N connection status 

21 may mean that a participant can speak and listen to the conference. The bridge 150 

22 associated with the conference, in this case bridge B3, grabs the decoded information 126 

23 (FIG. IB) from each decoder 122 (FIG. IB) associated with one of the participants 1, 2, 3, 4, 

24 5, and 6 so that the decoded information of these participants can be mixed with the other 
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1 audio signals of the conference, and participants 1, 2, 3, 4, 5, and 6 can be heard. For each 

2 participant 1, 3, 5, and 6, the encoder 124 (FIG. IB) associated with that participant grabs 

3 the mixed output 128 (FIG. IB) of the bridge (B3) so that participants 1, 3, 5, and 6 can 

4 listen to the conference. 

5 [0041] In conference A, the connection status of participants 2 and 4 (the French 

6 speaking persons) is "Speak" (S). The S connection status means that participants 2 and 4 

7 can be heard in the conference but cannot listen to the conference. For each participant 2 

8 and 4, the bridge 150 associated with the conference, in this case bridge B3, grabs the 

9 decoded information 126 from the decoder 122 associated with that participant so that the 

10 decoded information can be mixed with the other audio signals of the conference, and 

1 1 participants 2 and 4 can be heard. 

12 [0042] A second exemplary conference, conference B, uses bridge Bl . Conference B 

13 has six participants 1, 2, 3, 4, 5, and 7 that use codec numbers Cll, C21, C13, COS, C07, 

14 and CI 7, respectively, and are connected to the conference. In this conference, the 

15 connection status of participants 1, 3, and 5 (the English speaking individuals) is S. 

16 Consequently, participants 1, 3, and 5 can be heard in the conference but cannot listen to the 

17 conference. The bridge 150 associated with the conference, bridge Bl, grabs the decoded 

18 information 126 from the decoder 122 of participants 1, 3, and 5 so that their audio signals 

19 can be mixed with the rest of the audio signals of the conference allowing participants 1, 3, 

20 and 5 to be heard. 

21 [0043] Continuing with conference B, the connection status of participants 2, 4 (the 

22 French speaking individuals) and 7 (English to French translator) is "Normal" (N)- Thus, 

23 participants 2, 4, and 7 can both speak and listen to the conference. The speech of each one 

24 of the participants 2, 4, and 7 is enabled by the bridge Bl grabbing the decoded information 
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1 126 from the decoder 122 associated with those participants, so that the decoded 

2 information of these participants can be mixed with the other audio signals of the 

3 conference, and participants 2, 4, and 7 can be heard. The listening process of one of the 

4 participants is enabled by the encoder 124 associated with that participant grabbing the 

5 mixed output 1 28 of the bridge (B 1 ). 

6 [0044] The third exemplary conference, conference C, uses bridge B5, and has two 

7 participants 8 and 9, whom are using codecs having numbers C08 and C01, respectively. In 

8 this conference the connection status of both participants is "Normal" (N). 

9 [0045] In the fourth exemplary conference, conference D, bridge B4 is used. 

10 Conference D has four participants 8, 9, 10, and 11, who are using codecs numbered C08, 

11 C01, C10, and C04, respectively. In conference D, the connection status of participants 8 

12 and 9 is Speak (S). The S connection status means that participants 8 and 9 can be heard in 

13 the conference but cannot listen to the conference. For each participant 8 and 9, the bridge 

14 150 associated with the conference, in this case bridge B4, grabs the decoded information 

15 126 from the decoder 122 associated with that participant so that the decoded information 

16 can be mixed with the other audio signals of the conference, and participants 8 and 9 can be 

17 heard. The connection status of participants 10 and 1 1 is "Normal" (N)- Consequently, for 

18 each of participants 10 and 11, bridge B4, grabs the decoded information 126 from the 

19 decoder 122 associated with that participant, and the encoder 124 of that participant grabs 

20 the mixed output 1 28 of the bridge B4. 

21 [0046] The final exemplary embodiment, conference X, uses bridge Bz, and has 3 

22 participants N, N+I and N+2 who are using codecs numbered Cm, CI, and Ck, respectively. 

23 Conference X is a common conference where the connection status of all the participants is 

24 "Normal" (N). 
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1 [0047] The above example illustrates at least two special cases. The first case is a 

2 combination of two related conferences A and B, which is a conference of three English 

3 speaking individuals with a translator from French to English (FE) and French speaking 

4 individual with a translator from English to French (EF). 

5 [0048] The second case is the combination of two related conferences C and D, which 

6 can be a case in which two individuals 8 and 9 are discussing an issue among themselves 

7 while their peers 10 and 11 can listen to them. The peers 10 and 11 can thus have an 

8 internal discussion between themselves without being heard by the individuals 8 and 9. 

9 [0049] An instance of a participant in a conference may be a call to (or rather an 

10 instantiation of) an object representing a particular participant The existence of an instance 

11 of a participant implies that the participant's connection status in the conference is not 

12 'TSfone." In other embodiments, conferences may use connection statuses such as, but not 

13 limited to 5 "Exclusive" (E) and "Listening" (L). In E status, the participant is the only one 

14 that can be heard in the conference. Alternatively, L status allows the participant to listen to 

15 the conference without speaking. 

16 [0050] One or both connection statuses, E and L, may be used in an embodiment with 

17 any one of, any combination of, or all of connection statuses M, N, and None. For example, 

18 one embodiment may include connection statuses N, E, L, and None while another 

19 embodiment may include M, N, E, S, and None. 

20 [0051] In the present specification a connection change is any change in a current 

21 situation or configuration of the conference. The connection change can be, for example, to 

22 start a conference, terminate a conference, add a participant, or mute a participant. Based on 

23 the information embedded in the CCDB, the MCS 1 170 (FIG. 1 A) may perform connection 

24 changes by sending commands to the decoder 122 and the encoder 124 of the codec 120 that 
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1 determine which information to grab from the CACI 110 (FIG. 1A) and/or the DACI 140 

2 (FIG. IB), respectively, and/or by sending commands to the bridge 150 determining which 

3 information to grab from the DACI 140, 

4 [0052] The present invention is not limited to any specific type of database, and may use 

5 types of databases other than the ones explicitly disclosed above. For example, the database 

6 CCDB may include a bank of databases, having one database for each participant The 

7 database of a participant may have the participant's connection status in each conference. 

8 Alternatively, or in addition to this bank of databases, CCDB may have a bank of databases 

9 having one database for each conference. Each database may include the participants that 

10 are involved with that conference. The various databases may be related to each other 

1 1 enabling the controller to move from one database to the other. 

12 [0053] FIG. 3 is a flowchart showing the steps of an exemplary method 300 for 

13 disconnecting a participant The method 300 updates the conference connections that are 

14 affected by the participant disconnecting. Further, the method 300 ensures that the 

15 participant is no longer connected to any conference. Also, a participant being disconnected 

16 can place a conference in a state where the conference should be terminated (because, for 

17 example, there is only one participant left in that conference). Consequently, the method 

18 300 also ensures that any conference that should be terminated as a result of the 

1 9 disconnection is actually terminated. 

20 [0054] When the MCS 1170 (FIG. 1A) receives a control signal 1174 (FIG. la), either 

21 from the host 1200 (FIG. 1A) or from NI 1142 (FIG. 1A), that a participant has been 

22 disconnected, in step 310, the MCS 1170 starts a disconnection routine associated with the 

23 method 300. In step 320, the MCS 1170 searches for the CCDB entry related to the 
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1 participant that has been disconnected. In this step the appropriate row in the CCDB of FIG. 

2 2, for example, is found. 

3 [0055] Then in step 330, the MCS 1170 starts the conference loop, which is a loop 

4 including all conferences that are currently managed by the MCU. The loop may be 

5 searched conference by conference. For example, MCS 1170 may check how many 

6 conferences are currently managed by the MCU and may store this parameter, K. Then the 

7 MCS 1 170 may give the value 1 to a variable j and move along the appropriate row, in the 

8 CCDB of FIG. 2, for example, to the first conference found. 

9 [0056] In step 340, the MCS 1170 may check if the connection status of the 

10 disconnected participant in the current conference is "None." If it is different than "None," 

11 then the MCS 1 170 moves to step 350. Otherwise, MCS 1 170 moves to step 385. In step 

12 350, the MCS 1 170 may update the CCDB of FIG. 2, for example, with the new situation, 

13 (e.g., changes the connection status of the participant to "None"). Subsequently, in step 360 

14 MCS 1 1 70 may send signals to the rest of the participants in the conference indicating that 

1 5 one of the participants has been disconnected. 

16 [0057] Then, the MCS 1 170 may perform a check, step 370, for whether a termination 

17 condition is met The termination condition may occur when there are less than two 

18 participants. Alternatively, the termination condition may occur when the number of 

19 participants falls below a predetermined threshold. The predetermined threshold may be 

20 determined by financial or other considerations, for example. The termination condition 

21 may include a request to terminate the conference, which may be required in addition to or 

22 instead of the number of participants falling below a predetermined number. Termination of 

23 a conference can be done by the method described in conjunction with FIG. 4. The 

24 teimination condition may be a logical function of several termination conditions, and may 
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1 change dynamically according to a termination policy. If the termination condition is met, 

2 the MCS 1170 may move to step 365 and terminate the conference. However, if the 

3 termination condition is not met (for example, if there are more than two participants), the 

4 MCS 1 1 70 enables the continuation of the conference and moves to step 385. 

5 [0058] In step 385, the MCS 1 170 may increase the variable j by 1, and check, in step 

6 390, if an exit condition for exiting the loop is met. For example, an exit condition may be a 

7 condition indicative of all conferences being checked (e.g., j=K). The MCS 1170 then 

8 moves to step 395 and exits the loop, thereby, terminating the disconnection method. 

9 Otherwise, in step 397 MCS 1 170 moves to the next conference in the row and returns to 

10 step 340. Although in the example of the method 300 the participant is disconnected from 

11 all the conferences, a similar method could be used for disconnecting the participant from all 

12 but one or more specified conferences. 

13 [0059] FIG. 4 is a flow diagram illustrating the steps of an exemplary method 400 for 

14 terminating a conference (i.e., step 365 of FIG. 3). The method 400 notifies all participants 

15 of a conference that the conference is being terminated. If a participant is connected to the 

16 conference, and if the situation of the participant meets the disconnection criteria, the 

17 method 400 also disconnects the participant. After terminating the conference, the CCDB 

18 needs to be updated so that MCS 1 170 (FIG. 1 A) no longer sends control signals related to 

19 the terminated conference. Consequently, after notifying and disconnecting the participants, 

20 as appropriate, the method 400 updates the CCDB to reflect the conference being 

21 disconnected. 

22 [0060] When the MCS 1 170 receives a command to terminate a conference in step 410, 

23 the MCS 1 170 may start the termination routine of method 400. In step 420, the MCS 1 170 
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1 searches for the CCDB entry related to the conference to be terminated. For example, this 

2 step finds the appropriate column in the CCDB of FIG. 2, 

3 [0061 J Then in step 430, the MCS 1 170 starts the participant loop, which may be a loop 

4 that includes all participants that are currently connected to the conference. The MCS 1 170 

5 may check how many participants are currently connected to the conference and store the 

6 number of participants (parameter P). Then MCS 1170 (FIG. 1) may give a value I to a 

7 variable j, and move along the appropriate column (or conference) to the first participant. 

8 Then in step 435, the MCS 1 170 updates the CCDB, with the new conference state (e.g., the 

9 connection status of the participant, which is associated with the variable j, is changed to 

10 "None"). 

1 1 [0062] In step 440, the MCS 1 1 70 checks for a termination condition for the participant 

12 j. For example, a termination condition may be when the participant is not connected to any 

13 other conference. The check may be performed by observing the corresponding row of the 

14 participant in the CCDB. Unless the participant has any relation to or any instance in any 

15 other conference, then the termination condition of participant j is met. In this example, if 

16 the participant has any instance in any other conference, then the MCS 1 170 moves to step 

17 445 5 else the MCS 1 170 moves to step 450. In step 450, the MCS 1 170 sends an indication 

18 to participant j that the conference is terminated and in step 455 disconnects participant j. In 

19 another exemplary method, the MCS 1 170 may have another termination condition or policy 

20 to disconnect a participant The termination condition may be dynamic and may require a 

21 request for termination for the participant and/or other entity in addition to or as an 

22 alternative to the participant having a status of "None" in all conferences. If the termination 

23 condition is not met, for example, if the participant is connected to any other conference 
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1 440, then the MCS 1 170 sends the operator and/or the participant an indication, in step 445, 

2 that the current conference is terminated and the method 400 moves to step 465 . 

3 [0063] In step 465, the MCS 1170 may increase variable j by 1 and check, in step 470, 

4 whether j=P. If j=P, the MCS 1170 may move to step 477 and remove the appropriate 

5 column from the CCDB. Next, the method 400 moves to step 480, and finishes the 

6 conference termination. Otherwise (if #P), the MCS 1 170 will return to step 435 where the 

7 next participant in the column is processed. 

8 [0064] Those skilled in the art will appreciate that the MCS 1 170 can be any one of or 

9 any combination of software, hardware, and/or firmware. If the MCS 1170 is software, the 

10 MCS 1170 may reside in the host computer of the associated MCU. Alternatively, if the 

1 1 MCS 1 1 70 is hardware, the MCS 1 1 70 may be an additional unit located in the MCU. 

12 [0065] Furthermore, those skilled in the art will appreciate that the CCDB can be 

13 implemented in a single database or in several related databases. For example, the CCDB 

14 may be located in a bank of databases having one database per each participant, which may 

15 include the connection status of said participant in every conference that is currently 

1 6 controlled by the MCU and/or the connection parameters of the participant 

17 [0066] In the description and claims of the present application, each of the verbs, 

18 "comprise, 11 "include, 11 "have," and conjugations thereof, are used to indicate that the object 

19 or objects of the verb are not necessarily a complete listing of members, components, 

20 elements or parts of the subject or subjects of the verb. The present invention has been 

21 described using detailed descriptions of methods thereof that are provided by way of 

22 example and are not intended to limit the scope of the invention. The described methods 

23 comprise different features, not all of which are required in all embodiments of the 

24 invention. Some embodiments of the present invention utilize only some of the features or 
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1 possible combinations of the features. Variations of embodiments of the present invention 

2 that are described and embodiments of the present invention having different combinations 

3 of features noted in the described embodiments will occur to persons of the art. The scope of 

4 the invention is limited only by the following claims. 
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WHAT IS CLAIMED 



1 LA method for controlling conferences of a plurality of participants having at least one 

2 participant that can participate in two or more conferences simultaneously, the method 

3 comprising: 

4 creating a cross conference database, the cross conference database including selected 

5 connection parameters and selected connection statuses; and 

6 performing a connection change based on the cross conference database. 

1 2. The method of claim 1 , wherein performing a connection change comprises: 

2 determining how the connection change affects each of the conferences and the 

3 participants. 

1 3. The method of claim 2, wherein performing a connection change comprises: 

2 updating the cross conference database according to a result of the determining. 

1 4. The method of claim 2, wherein performing a connection change comprises: 

2 implementing a result of the updating. 

1 5. The method of claim 1, wherein performing connection change comprises: 

2 detecting all participants and all conferences involved in the connection change; 
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3 



determining how the connection change affects each of the conferences and the 



4 



5 



participants; 

ipdating the cross conference database according to a result of the determining; and 



6 



implementing a result of the updating. 



1 6. The method of claim 1 further comprising notifying each of the participants affected by the 

2 connection change about the connection change before performing the connection change, 

1 7. The method of claim 1 wherein the connection parameters include a parameter associated 

2 with a codec that is associated with each participant. 

1 8. The method of claim 1 wherein the connection parameters include a parameter associated 

2 with a bridge that is associated with each conference. 

1 9. The method of claim 1 wherein at least one of the connection statuses is associated with a 

2 participant in the conferences. 

1 10. The method of claim 1 wherein the connection status is capable of having statuses of listen. 

1 11. The method of claim 1 wherein the connection status is capable of having statuses of 

2 exclusive. 
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1 12. The method of claim 1 wherein the connection status is capable of having statuses of mute. 

1 1 3 . The method of claim 1 wherein the connection status is capable of having statuses of force. 

1 14. The method of claim 1 wherein the connection status is capable of having statuses of 

2 normal. 

1 15. The method of claim 1 wherein the connection status is capable of having statuses of none. 

1 16. The method of claim 1 wherein the connection status is capable of having statuses of 

2 normal, none, exclusive, mute, force, speak, and listen. 

1 17. The method of claim 1 wherein the conference is an audio conference. 

1 18. The method of claim 1 wherein the conference is a multimedia conference. 
1 
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1 19. A method comprising conducting a conference by: 

2 receiving audio signals from a plurality of endpoints; 

3 broadcasting the audio signal onto an audio common interface; 

4 selecting at least one audio signal to be mixed, the selecting is performed 

5 using a cross conference reference table to determine a selection; 

6 mixing the selected audio signal; and 

7 transmitting the mixed signal to at least one end point; 
wherein at least one participant can participant in two or more conferences. 

1 20. A method comprising: 

2 creating a cross conference database; 

3 establishing at least one conference having at least one participant that 

4 participates in at least one other conference by 

5 updating a cross connection database; and 

6 conducting the conference by 

7 receiving audio signals from a plurality of endpoints, 

8 broadcasting the audio signal onto an audio common interface, 

9 selecting at least one audio signal to be mixed, the selecting is performed 

10 using a cross conference reference table to determine a selection, 

1 1 mixing the selected audio signal, and 

12 transmitting the mixed signal to at least one end point. 
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1 21. The method of claim 20 wherein the receiving farther comprises processing the received 

2 audio signals, the processing is performed using the cross conference database to determine 

3 which module performs a process. 

1 22. The method of claim 21 wherein the audio signals that were received are encoded. 

1 23. The method of claim 22 wherein the processing the received audio signals includes 

2 decoding of the received audio signals. 

1 24. The method of claim 23 wherein processing the received audio signals includes analyzing 

2 the decoded audio signals. 

1 25. The method of claim 21 wherein transmitting the mixed signal is performed based on the 

2 cross conference database. 

1 26. The method of claim 21 wherein the module is a codec. 

1 27. The method of claim 20 wherein performing a connection change based on the cross 

2 conference database, including: 

3 detecting all participants and all conferences involved in the connection change, 

4 determining how the connection change affects each of the conferences and the 

5 participants, 



30 



WO 02/091641 



PCT/1L02/00361 



updating the cross conference database according to a result of the determining, 
implementing a result of the updating. 
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1 28. A multipoint control unit comprising: 

2 a management and control system that manages the participation of at least one 

3 participant in two or more conferences. 

1 29. The multipoint control unit of claim 28 wherein the management and control system 

2 includes a cross conference database. 

1 30. The multipoint control unit of claim 28 wherein two or more of the conferences are related 

2 to one another. 
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1 31. A system comprising: 

2 a multipoint control unit having a management and control system for managing and 

3 controlling two or more related conferences. 

1 32. The system of claim 31 wherein two of the conferences are related by having at least one 

2 participant participating in both conferences. 



1 33. The system of claim 3 1 wherein the system 

2 creates a cross conference database, the cross conference database including selected 

3 connection parameters and selected connection statuses. 

1 34. The system of claim 33 wherein the system 

2 performs a connection change based on the cross conference database by 

3 detecting all participants and all conferences involved in the connection change, 

4 determining how the connection change affects each of the conferences and the 

5 participants, 

6 updating the cross conference database according to a result the determining, and 
- 7 implementing a result of the updating. 

1 35. The system of claim 32 wherein the management control system 
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2 notifies each of the participants affected by the connection change about the connection 

3 change before performing the connection change. 

1 36. The system of claim 3 1 further comprising a codec. 
1 

2 37. The system of claim 36 wherein the codec 

3 receives signals from the compressed audio common interface, 

4 decodes the signals, wherein the receiving is done using the cross conference 

5 database, and 

6 broad casts the signals that have been decoded onto the decoded audio common 

7 interface. 

1 38. The system of claim 36 wherein the codec 

2 receives a mixed output from the decoded audio common interface, 

3 encodes the mixed output, and 

4 broadcasts the mixed output that was encoded onto a compressed audio common 

5 interface. 



1 39. The system of claim 3 1 further comprising a bridge. 
1 
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2 40. The system of claim 39 wherein the bridge mixes a selected group of decoded audio signals 

3 associated with the conference to produce a mixed output, wherein the selection is done 

4 using the cross connection database, 

1 4 1 . The system of claim 3 1 further comprising: 

2 a decoded audio common interface. 

1 42. The system of claim 31 further comprising: 

2 a compressed audio common interface. 
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1 43. A system controlling audio or multimedia conferences of a plurality of participants having 

2 at least one participant that can participate in two or more conferences, the method comprising: 

3 means for creating a cross conference database, the cross conference database including 

4 connection parameters and connection statuses; and 

5 means for performing a connection change using the cross conference database. 

1 44. A method for making a system for controlling audio or multimedia conferences of a 

2 plurality of participants having at least one participant that can participate in two or more 

3 conferences the method comprising: 



4 providing a management and controlling system within a multipoint control unit, the 

5 management and control system creating a cross conference database, the cross 

6 conference database including connection parameters and connection statuses; 

7 and 

8 configuring the multipoint control unit to perform a connection change based on the 

9 cross conference database. 
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(57) Abstract: The present invention is a system and a method for providing a control unit for a multipoint multimedia/audio confer- 
ence that enables one or more participants (1 -N+2) to take part in more than one conference (A-X) simultaneously. The control unit 
can operate in audio and/or video sections of an MCU and/or Management and Control System (MCS). The MCS, in an exemplary 
embodiment of the present invention, controls the participation of at least one participant in more than one conference simultaneously 
by using a Cross -Conference Database (200). The MCU performs connection changes affecting which participants are associated 
with one or more conferences based on the information that is stored in the CCDB. 
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